How do potential donors respond to ‘impact per dollar’ information?

Dr. David Reinstein (University of Exeter, innovationsinfundraising.org), Dr. Robin Bergh (Harvard), Donor’s Voice advisory board

01/10/2019

How do potential donors respond to ‘impact per dollar’ information?

Empathic and Numerate Giving: Donor responses to charity effectiveness and impact

Dr. David Reinstein, University of Exeter, innovationsinfundraising.org
with Dr. Robin Bergh (Harvard), Donor’s Voice advisory board

Does “Impact Matter” to donors?

Impact Matters .org

https://www.givewell.org
Organizations like ‘GiveWell’ and ‘ImpactMatters’ are promoting “impact, e.g., ‘lives saved/$’”


\(\rightarrow\) How will potential donors respond to this ‘analytical’ information?

Energise effective giving? or Switch off empathy?

\(\rightarrow\) (How and when) should charities advertise their impact/$?

Outline

  1. The question

  2. What previous research suggests

  1. What we’ve done and what we’ve found

  2. What we’d like to do

  1. Other projects: InnovationsInFundraising.org, GiveIfYouWin.org

What do we mean by ‘impact/effectiveness’?

\(B(G_i)\): Beneficial outcome achieved by charity \(i\) with total donations \(G_i\)


ULTIMATE outcome

  • Lives saved, quality-adjusted-life-years added, etc.
  • Not ‘output’ – ‘nets provided’ nor ‘paintings purchased’


Actually, “output per $” \(\times\) “impact per output”

  • Actually, the marginal impact of each $ donated on this

… what GiveWell and ImpactMatters attempt to measure; Strong evidence that this varies greatly!

‘This kind of effectiveness’ (impact)

From GiveWell calculations:

(NOT ‘this kind of effectiveness’)

Theory: should ‘impact’ drive giving or shut-down empathy?

Behavioral Economics idea/ideal: “impact-altruism”

‘Innumerate empathy?’… cognitive biases, barriers to effectiveness

‘Dual process’ - fast/slow thinking; ‘spontaneous generosity vs. deliberative self-interest?’

Relational models … ‘market vs sharing norms’

‘Motivated reasoning’, excuses not to give; double-edged

‘Signaling’ empathy vs effectiveness

The evidence: prior studies

Claim: Better to portray an individual (child) than convey the total affected

Small & Loewenstein (03); Small et al (07); Kogut & R (05)

Driven by System-1 empathy, switched off by analytic thinking


Small, Lowenstein, Slovic (2007):

[Study 3] “individuals who faced an identifiable victim donated more…”

“…than those who faced victim statistics, p < .01,”

…“than those who faced [both] an identifiable victim [and] statistics, p < .05.”

Small et al, ’07, Study 4


Priming analytic thinking reduced donations to an identifiable victim relative to a feeling-based thinking prime.


Yet, the primes had no distinct effect on donations to statistical victims, which is symptomatic of the difficulty in generating feelings for these victims.


Tightly bounded null, but … nonlinearity?

Ratings and information in general: mixed evidence

  • Yörük (2016, JEMS): RD w/ Charity Navigator; significant for ‘small’ charities only

    • See also Brown ea (2017), Gordon ea (2009)

Parsons (2007)

2 x 2 mailing appeal for People with Aids Coalition-Houston

  • Add “Service efforts and accomplishment info”(SEA)
  • Add favorable “FINANCIAL” spending/overhead ratio info

FINANCIAL (alone) \(\rightarrow\) 180% increase in odds of donating among prior donors (\(p<0.05\))

Karlan and Wood (2017)

Adding scientific impact to appeal (& removing emotional text):

\(\rightarrow\) little net effect

\(\rightarrow\) reduced (increased) giving among small (large) prior donors


But some potential confounds…

Bergh/Reinstein/DV studies

7 giving experiments, various contexts

Bergh/Reinstein/DV studies

  • Actual charitable giving/asks

  • Field and ‘M-Turk’ settings with realistic contexts

2x2:

  • Measures of charity efficiency/effectiveness

  • Emotional-empathy-inducing images

Bergh project (Mturk)

Context (for studies 1-5)

  • Amazon’s Mechanical-turk participants (Americans)

\(\rightarrow\) final sample sizes: 398, 614, 611, 608, and 433 in Studies 1-5 respectively (variation tracking design complexity)

Payments: $1.50 ($2 in s4) baseline, Bonuses: $3 in S1-S2, $5 in S5, Raffle: $50 (1:25 odds) in S3-S4

Donation asks (from bonus) & treatments: 1 charity (or 2 in same category): Syria relief, Polio

Bergh/Reinstein setups

Studies 1-3:

Study 4:

Study 4

Bergh/Reinstein Results

Overall:


Strong positive effect of images on donation incidence, amounts


Little impact of effectiveness information on donation incidence, amounts

  • “Fairly tightly-bounded null”

More on Bergh/Reinstein, Essex

(Study 6: field context)

Essex context

Connected to EssexLab 2019 Omnibus online survey

  • \(\approx\) 20 minutes, many psycho/demog/polit/econ questions, mostly unrelated to charity
  • Completion \(\rightarrow\) raffle for 1 of 20 Amazon vouchers worth £50 each

  • Given information about blindness in general

  • (Conditional) donation part

The ‘last’ Omnibus questions just before this

2 \(\times\) 2, “balanced blocked randomisation”, “between-subject”:

  1. Analytic information about ‘cost per outcome’ & ‘cost per impact’


  1. Empathy-inducing image: picture of blind girl (Yes/No)

Introduction screen (all)

Image treatment (Half of participants)

Control: description, choice (1/2 of subjects)

Info-treatment: description, choice (1/2 of subjects)

Donation amount choices

Donation slider
Donation slider

Results

Limited power to detect differences in amounts donated or incidence by treatment


Image reduced donation (incidence and amounts) to Guide Dogs (fairly strong and significant effect)

And ‘increased’ donations to River Blindness, but not statistically significantly)

Levels

Histogram: by image treatment 
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

Histogram: by information treatment 
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

—-

Donation incidence

treat_image treat_eff_info donated don_gd don_river n
Image Info 0.537 0.168 0.368 95
Image No info 0.505 0.147 0.358 95
No image Info 0.511 0.213 0.298 94
No image No info 0.532 0.245 0.287 94

Fisher tests: some donation

estimate p.value conf.low conf.high method alternative Experiment
0.919 0.884 0.498 1.69 Fisher two.sided Effectiveness vs control
1.134 0.772 0.617 2.09 Fisher two.sided Effectiveness| Image present
1.021 1.000 0.669 1.56 Fisher two.sided Effectiveness (pooled)

Donation incidence

treat_image treat_eff_info donated don_gd don_river n
Image Info 0.537 0.168 0.368 95
Image No info 0.505 0.147 0.358 95
No image Info 0.511 0.213 0.298 94
No image No info 0.532 0.245 0.287 94
estimate p.value conf.low conf.high method alternative Experiment
1.05 0.91 0.67 1.6 Fisher two.sided Effectiveness: RB
0.97 1.00 0.56 1.7 Fisher two.sided Effectiveness: GD
1.38 0.16 0.88 2.2 Fisher two.sided Image: RB
0.63 0.09 0.36 1.1 Fisher two.sided Image: GD

Fisher tests: donated to Guide dogs/River blindness

Simple tests

By treatment combination

By charity, by treatment (combination)

Donations to Guide Dogs by treatment combination

Donations to River Blindness by treatment combination

Donor Voice trial

Presentation coming up soon.

Donor Voice trial

Spoiler warning:

Some positive effect of impact-related information

Sum up

Early take-aways: Don’t fear the info

“Images work”

Image treatments can be effective at motivating giving, including impactful, efficient giving


“Info does not hurt, may help”

Analytic information about impact:

  • Does not seem to (strongly) decrease donations

  • May increase donations in some cases (Donor’s voice mailing; Karlan & W, Parsons work)

  • Mixed evidence on ‘efficiency info dampening the impact of identified victim images’

\(\rightarrow\) this may not be a ‘major barrier’ to promoting effective giving

Caveats: More power needed, cannot rule out substantial effects

Future work (with you?)

Larger-scale field trials examining responses to real impact-per-$ information, in various presentations


Through your standard platforms

Digital, social-media, advertisements on your behalf

  • (co)-financed with our research grants!

innovationsinfundraising.org

Connect and communicate between and among researchers, practitioners and supporters. Curate, organize, and evaluate the rigorous evidence on ‘what works’.

innovationsinfundraising.org

giveifyouwin.org

‘Give if you win’: a new way to promote giving.